Sample Efficient Optimization for Learning Controllers for Bipedal Locomotion

Learning policies for bipedal locomotion can be difficult, as experiments are expensive and simulation does not usually transfer well to hardware. To counter this, we need al- gorithms that are sample efficient and inherently safe. Bayesian Optimization is a powerful sample-efficient tool for optimizing non-convex black-box functions. However, its performance can degrade in higher dimensions. We develop a distance metric for bipedal locomotion that enhances the sample-efficiency of Bayesian Optimization and use it to train a 16 dimensional neuromuscular model for planar walking. This distance metric reflects some basic gait features of healthy walking and helps us quickly eliminate a majority of unstable controllers. With our approach we can learn policies for walking in less than 100 trials for a range of challenging settings. In simulation, we show results on two different costs and on various terrains including rough ground and ramps, sloping upwards and downwards. We also perturb our models with unknown inertial disturbances analogous with differences between simulation and hardware. These results are promising, as they indicate that this method can potentially be used to learn control policies on hardware.Comment: To appear in International Conference on Humanoid Robots (Humanoids '2016), IEEE-RAS. (Rika Antonova and Akshara Rai contributed equally

Deep Kernels for Optimizing Locomotion Controllers

Sample efficiency is important when optimizing parameters of locomotion controllers, since hardware experiments are time consuming and expensive. Bayesian Optimization, a sample-efficient optimization framework, has recently been widely applied to address this problem, but further improvements in sample efficiency are needed for practical applicability to real-world robots and high-dimensional controllers. To address this, prior work has proposed using domain expertise for constructing custom distance metrics for locomotion. In this work we show how to learn such a distance metric automatically. We use a neural network to learn an informed distance metric from data obtained in high-fidelity simulations. We conduct experiments on two different controllers and robot architectures. First, we demonstrate improvement in sample efficiency when optimizing a 5-dimensional controller on the ATRIAS robot hardware. We then conduct simulation experiments to optimize a 16-dimensional controller for a 7-link robot model and obtain significant improvements even when optimizing in perturbed environments. This demonstrates that our approach is able to enhance sample efficiency for two different controllers, hence is a fitting candidate for further experiments on hardware in the future.Comment: (Rika Antonova and Akshara Rai contributed equally

Bayesian Optimization Using Domain Knowledge on the ATRIAS Biped

Controllers in robotics often consist of expert-designed heuristics, which can be hard to tune in higher dimensions. It is typical to use simulation to learn these parameters, but controllers learned in simulation often don't transfer to hardware. This necessitates optimization directly on hardware. However, collecting data on hardware can be expensive. This has led to a recent interest in adapting data-efficient learning techniques to robotics. One popular method is Bayesian Optimization (BO), a sample-efficient black-box optimization scheme, but its performance typically degrades in higher dimensions. We aim to overcome this problem by incorporating domain knowledge to reduce dimensionality in a meaningful way, with a focus on bipedal locomotion. In previous work, we proposed a transformation based on knowledge of human walking that projected a 16-dimensional controller to a 1-dimensional space. In simulation, this showed enhanced sample efficiency when optimizing human-inspired neuromuscular walking controllers on a humanoid model. In this paper, we present a generalized feature transform applicable to non-humanoid robot morphologies and evaluate it on the ATRIAS bipedal robot -- in simulation and on hardware. We present three different walking controllers; two are evaluated on the real robot. Our results show that this feature transform captures important aspects of walking and accelerates learning on hardware and simulation, as compared to traditional BO.Comment: 8 pages, submitted to IEEE International Conference on Robotics and Automation 201

Can Electronic Notebooks Enhance the Classroom?

We are interested in prototyping future computing environments that will enhance the classroom experience and empower both teacher and student. In this paper, we describe the Classroom 2000 project at Georgia Tech which is integrating personal and group pen-based technology, audio services and the World-Wide Web to record in-class interactions for later review

Predicting human interruptibility with sensors, in

A person seeking someone else’s attention is normally able to quickly assess how interruptible they are. This assessment allows for behavior we perceive as natural, socially appropriate, or simply polite. On the other hand, today’s computer systems are almost entirely oblivious to the human world they operate in, and typically have no way to take into account the interruptibility of the user. This paper presents a Wizard of Oz study exploring whether, and how, robust sensor-based predictions of interruptibility might be constructed, which sensors might be most useful to such predictions, and how simple such sensors might be. The study simulates a range of possible sensors through human coding of audio and video recordings. Experience sampling is used to simultaneously collect randomly distributed self-reports of interruptibility. Based on these simulated sensors, we construct statistical models predicting human interruptibility and compare their predictions with the collected self-report data. The results of these models, although covering a demographically limited sample, are very promising, with the overall accuracy of several models reaching about 78%. Additionally, a model tuned to avoiding unwanted interruptions does so for 90 % of its predictions, while retaining 75 % overall accuracy

Interpersonal interactions for haptic guidance during balance exercises

Background: Caregiver–patient interaction relies on interpersonal coordination during support provided by a therapist to a patient with impaired control of body balance. Research question: The purpose of this study was to investigate in a therapeutic context active and passive participant involvement during interpersonal support in balancing tasks of increasing sensorimotor difficulty. Methods: Ten older adults stood in semi-tandem stance and received support from a physical therapist (PT) in two support conditions: 1) physical support provided by the PT to the participant’s back via an instrumented handle affixed to a harness worn by the participant (“passive” interpersonal touch; IPT) or 2) support by PT and participant jointly holding a handle instrumented with a force-torque transducer while facing each other (“active” IPT). The postural stability of both support conditions was measured using the root-mean-square (RMS) of the Centre-of-Pressure velocity (RMS dCOP) in the antero-posterior (AP) and medio-lateral (ML) directions. Interpersonal postural coordination (IPC) was characterized in terms of cross-correlations between both individuals’ sway fluctuations as well as the measured interaction forces. Results: Active involvement of the participant decreased the participant’s postural variability to a greater extent, especially under challenging stance conditions, than receiving support passively. In the passive support condition, however, stronger in-phase IPC between both partners was observed in the antero-posterior direction, possibly caused by a more critical (visual or tactile) observation of participants’ body sway dynamics by the therapist. In-phase cross-correlation time lags indicated that the therapist tended to respond to participants’ body sway fluctuations in a reactive follower mode, which could indicate visual dominance affecting the therapist during the provision of haptic support. Significance: Our paradigm implies that in balance rehabilitation more partnership-based methods promote greater postural steadiness. The implications of this finding with regard to motor learning and rehabilitation need to be investigated